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APPELLANT'S BRIEF ON APPEAL 

Appellants hereby appeal to the Board of Patent Appeals and Interferences 
from the Examiner's Final Rejection of claims 91-130 which was contained in the 
Office Action mailed. 

A timely Notice of Appeal was filed July 6, 2004. 

Real Party In Interest 

As indicated above in the caption of the Brief, the Eastman Kodak 
Company is the real party in interest. 

Related Appeals And Interferences 

No appeals or interferences are known which will directly affect or be 
directly affected by or have bearing on the Board's decision in the pending appeal. 

Status Of The Claims 

Claims 91-152 are pending in the application. 
Claims 1-90 are cancelled. 

Claims 131-152 stand withdrawn from consideration. 

This appeal is directed to Claims 91-130 which stand finally rejected by 
the Office Action dated April 6, 2004. Appendix I provides a clean, double 
spaced copy of the claims on appeal. 

Status Of Amendments 

No amendments were filed subsequent to the Final Rejection. 

Summary Of The Invention 

Applicant's invention is directed to the automatic identification of the 
main subject of an individual image and, automatically altering the identified 
region of the image to emphasize the identified region. 

As noted in the prior art, it was known in the motion picture industry, at 
the time of the present invention, to manually identify the main subject of an 
image or frame. After the main subject had been manually identified, such as by 
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manually outlining the subject of interest, the color values were changed to 
emphasize the identified region. The process was labor intensive and limited by 
cost and inconvenience. 

To overcome this problem, the applicant's apparatus as defined in the 
Claims, to the effect that the main subject of an image is automatically identified 
so that the pixel values of the image can then be automatically altered to 
emphasize the main subject. 

Identification of the main subject may be accomplished by 
segmentation of the image into regions. A main subject belief map of the regions 
may be generated. The belief map may be generated by calculation of saliency 
level. 

Pixel alteration may involve changing pixel color saturation, hue, 
luminance, blur, saturation, or inversion. The pixel values of the main subject or 
the pixel values that are not a part of the main subject may be altered. 

Issues For Review By The Board 

The following issue is presented for review by the Board of Patent Appeals 
and Interferences: Are Claims 91-130 unpatentable as being anticipated by Luo et 
al. 6,504,951 under 35 U.S.C 102(e)? 

Grouping Of Claims 

The patentability of claim 91-130, for purposes of this appeal only, stands 
or falls together. 
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Arguments 

The Rejection 

In rejecting Claims 91-130, the Examiner proposes that Luo et al. teaches 
automatically identifying a main subject, which the Examiner suggests is the sky. 

The Examiner finds that the pixel values are automatically altered to 
emphasize the main subject (the sky) by eliminating pixels having a texture above 
a threshold or by filling voids of connected components. The Examiner states that 
the pixel altering (thresholding, eliminating and region growing) are conducted 
after identifying the main subject, the sky. 
The Arguments 

First, it should be noted that the following comments interpreting the Luo 
et al. *951 reference come from Jiebo Luo, one of the inventors of the present 
application and the first-named inventor of the reference. Accordingly, it is felt 
that the interpretation of the disclosure of the reference should be given 
exceptional weight. 

Luo et al. does not disclose automatically identifying a main subject of the image: 
Luo et al. '951 teaches a procedure for detecting a sky region in an image. 
If an image is determined to include a sky, it is a strong indicator that the image is 
an outdoor scene. This information my be used as a guide for manipulation tasks 
such as orientation of the image or excluding the sky region in detecting the main 
subjects in the image (col. 1 , lines 48-52). 

Accordingly, early on we see that the Examiner's contention that Luo et al. 
discloses a method for "automatically identifying a main subject (candidate sky) 
of the image" (Final Rejection pg. 3, Paragraph 7, line 5) is incorrect. In fact, Luo 
et al. specifically states "in detecting main subjects in the image, sky regions can 
usually be excluded because they are likely to be part of the background." (col. 1, 
lines 48-52). 

Luo et al. does not disclose automatically altering pixel values of said image to 
emphasize said main subject said altering following said identifying: 
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Even if the sky region detected by Luo et al. f 951 was in fact the main 
subject of the image, there is no disclosure of altering pixel values of the image to 
emphasize the sky following its identification. 

The Examiner contends that Luo et al's process of eliminating pixels 
having texture above a threshold value and the process of filling voids of 
connected components emphasize the sky. The Examiner further contends that 
these processes are done after the sky has been identified. Appellants traverse this 
contention. 

In fact, Luo et aFs steps of eliminating pixels having texture above a 
threshold or filling voids of connected components are steps towards finding sky 
regions because sky regions are smooth (and thus not textured). Since the process 
is intended to find sky regions, it by definition can not take place after the sky has 
been identified. The eliminating pixels step in Luo et al. is an intermediate step in 
finding sky regions. 

Further, the step of eliminating pixels having texture above a threshold or 
filling voids of connected components are not done to alter the pixel values for 
emphasizing the sky regions. Emphasizing a region relative to other regions does 
not mean, in the context of the present application, eliminating the other regions; 
it simply means that the other regions are not emphasized. 

Summary 

Luo et al. does not disclose automatically identifying a main subject of the 
image. Nor does Luo et al. disclose automatically altering pixel values of the 
image to emphasize the main subject, the altering following the identifying step. 
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Conclusion 

For the above reasons, Appellants respectfully request that the Board of 
Patent Appeals and Interferences reverse the rejection by the Examiner and 
mandate the allowance of Claims 91-130. 



Respectfully submitted, 



Robert Luke Walker/amb 




Telephone: (585) 588-2739 Registration No. 30,700 

Facsimile: (585)477-1148 

Enclosures 
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Appendix I - Claims on Appeal 

91. (Previously presented) A method for modifying an image 
having pixels, comprising the steps of: 

automatically identifying a main subject of the image, and 
automatically altering pixel values of said image to emphasize said main 
subject, said altering following said identifying. 

92. (Previously presented) The method of Claim 91 wherein said 
identifying further comprises segmenting said image into regions. 

93. (Previously presented) The method of Claim 92 wherein said 
identifying further comprises generating a main subject belief map of said regions. 

94. (Previously presented) The method of Claim 91 wherein said 
altering pixel values further comprises altering pixel color saturation. 

95. (Previously presented) The method of Claim 91 wherein said 
altering pixel values further comprises altering pixel hue. 

96. (Previously presented) The method of Claim 91 wherein said 
altering pixel values further comprises altering pixel luminescence. 

97. (Previously presented) The method of Claim 91 wherein said 
altering pixel values further comprises altering pixel blur. 

98. (Previously presented) The method of Claim 91 wherein said 
altering emphasizes said main subject by altering pixel values that are a part of 
said main subject. 
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99. (Previously presented) The method of Claim 91 wherein said 
altering emphasizes said main subject by altering pixel values that are not a part of 
said main subject. 

100. (Previously presented) The method of Claim 91 wherein said 
altering further comprises desaturating the pixels that are not a part of said main 
subject. 

101. (Previously presented) The method of Claim 100 wherein 
said desaturating further comprises: 

calculating the luminance values for the pixels that are not a part of said 
main subject, and 

replacing the color values of the pixels that are not a part of said main 
subject with respective luminance values. 

102. (Previously presented) The method of Claim 91 wherein said 
altering further comprises enhancing saturation of the pixels that are a part of said 
main subject. 

103. (Previously presented) The method of Claim 91 wherein said 
altering further comprises inverting the pixel hue values of the pixels that are a 
part of said main subject. 

104. (Previously presented) The method of Claim 91 wherein said 
altering further comprises inverting the pixel hue values of the pixels that are not a 
part of said main subject. 

105. (Previously presented) The method of Claim 91 wherein said 
identifying further comprises the steps of: 
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segmenting the image into a plurality of regions based on uniform image 
characteristics; 

calculating a level of saliency for said plurality of regions, and 
assigning a believe value to the pixels corresponding to said level of 
saliency. 

106. (Previously presented) A system for modifying an image 
having pixels, comprising the steps of: 

means for automatically identifying a main subject of the image, and 
means for automatically altering pixel values of said image to emphasize 
said main subject, said altering following said identifying. 

107. (Previously presented) The system of Claim 106 wherein 
said means for identifying further comprises means for segmenting said image 
into regions. 

108. (Previously presented) The system of Claim 1 07 wherein 
said means for identifying further comprises means for generating a main subject 
belief map of said regions. 

109. (Previously presented) The system of Claim 106 wherein 
said means for altering pixel values further comprises means for altering pixel 
color saturation. 

110. (Previously presented) The system of Claim 106 wherein 
said means for altering pixel values further comprises means for altering pixel 
hue. 
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111. (Previously presented) The system of Claim 106 wherein 
said means for altering pixel values further comprises means for altering pixel 
luminescence. 

112. (Previously presented) The system of Claim 106 wherein 
said means for altering pixel values further comprises means for altering pixel 
blur. 

113. (Previously presented) The system of Claim 106 wherein 
said means for altering emphasizes said main subject by altering pixel values that 
are a part of said main subject. 

114. (Previously presented) The system of Claim 106 wherein 
said means for altering emphasizes said main subject by altering pixel values that 
are not a part of said main subject. 

115. (Previously presented) The system of Claim 106 wherein 
said means for altering further comprises means for desaturating the pixels that are 
not a part of said main subject. 

116. (Previously presented) The system of Claim 115 wherein 
said means for desaturating further comprises: 

means for calculating the luminance values for the pixels that are not a part 
of said main subject, and 

means for replacing the color values of the pixels that are not a part of said 
main subject with respective luminance values. 
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117. (Previously presented) The system of Claim 106 wherein 
said means for altering further comprises means for enhancing saturation of the 
pixels that are a part of said main subject. 

118. (Previously presented) The system of Claim 106 wherein 
said means for altering further comprises means for inverting the pixel hue values 
of the pixels that are a part of said main subject. 

119. (Previously presented) The system of Claim 1 06 wherein 
said means for altering further comprises means for inverting the pixel hue values 
of the pixels that are not a part of said main subject. 

120. (Previously presented) The system of Claim 1 06 wherein 
said means for identifying further comprises the steps of: 

means for segmenting the image into a plurality of regions based on 

uniform image characteristics; 

means for calculating a level of saliency for said plurality of regions, and 
means for assigning a believe value to the pixels corresponding to said 

level of saliency. 

121. (Previously presented) A method of modifying an image 
having pixels, comprising the steps of: 

automatically generating one or more belief values, each said belief value 
being associated with one of a plurality of regions of the image, said belief values 
each being related to the probability that the associated region is a main subject of 
the image; 
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following said generating, automatically altering pixel values in said 
plurality of regions of said image, in accordance with said associated belief 
values. 

122. (Previously presented) The method of Claim 1 2 1 wherein 
said altering further comprises: 

determining a saturation value for each pixel; and 

altering the pixel saturation values according to said associated belief 

values. 

123. (Previously presented) The method of Claim 121 wherein 
said altering further comprises: 

determining a luminance value for each pixel; and 

altering the pixel luminance values according to said associated belief 

values. 

124. (Previously presented) The method of Claim 121 wherein 
said altering further comprises: 

determining a hue value for each pixel, and 

altering the pixel hue values according to said associated belief 

values. 

125. (Previously presented) The method of Claim 121 wherein 
said altering further comprises: 

determining a blur value for each pixel, and 

altering the pixel blur values according to said associated belief values. 
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126. (Previously presented) A method of modifying an image 
having pixels, comprising: 

automatically generating a main subject belief map containing values 
indicating the location of a plurality of regions in the image, said main subject 
belief map containing at least a first belief value associated with one of said 
plurality of regions, said belief values being related to the probability that the 
associated region is a main subject of the image; and 

automatically altering pixel values in said plurality of regions of said 
image, in relation to said associated belief values, said altering following said 
generating. 

127. (Previously presented) The method of Claim 126 wherein 
said altering further comprises: 

determining a saturation value for each pixel, and 

altering the pixel saturation values according to said associated belief 

values. 

128. (Previously presented) The method of Claim 1 26 wherein 
said altering further comprises: 

determining a luminance value for each pixel, and 

altering the pixel luminance values according to said associated belief 

values. 

129. (Previously presented) The method of Claim 126 wherein 
said altering further comprises: 

determining a hue value for each pixel, and 
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altering the pixel hue values according to said associated belief values. 

130. (Previously presented) The method of Claim 126 wherein 
said altering further comprises: 

determining a blur value for each pixel, and 

altering the pixel blur values according to said associated belief values. 
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